Optimal fusion of video and RF data for detection and tracking with object occlusion

Occlusions can degrade object tracking performance in sensor imaging systems. This paper describes a robust approach to object tracking that fuses video frames with RF data in a Bayes-optimal way to overcome occlusion. We fuse data from these heterogeneous sensors, and show how our approach enables tracking when each modality cannot track individually. We provide the mathematical framework for our approach, details about sensor operation, and a description of a multisensor detection and tracking experiment that fuses real collected image data with radar data. Finally, we illustrate two benefits of fusion: improved track hold during occlusion and diminished error.